A new hybrid structure of speech recognizer based on HMM and neural network

In this paper, we introduced a new framework of speech recognizer based on HMM and neural net. Unlike the traditional hybrid system, the neural net was used as a post processor, which classify the speech data segmented by HMM recognizer. The purpose of this method is to improve the top-choice accuracy of HMM based speech recognition system in our lab. Major issues such as how to use the segmentation information of HMM in neural net, the structure of the neural net, the choice of the error metric for training neural net, and the determination of the training procedure are investigated within a set of experiments. In these experiments, we attempt to recognize 68 phoneme like units in continuous speech. Our results indicate that this is a potential method. About 20% can be obtained to improve the recognition accuracy for multi-speaker system in syllable level, and 10% for speaker independent system.